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Volume II reviews the history of oxygen 
tank no. 2 on its cryogenic shelf as launched 
in Spacecraft 109. The tank assembly, space- 
craft installation, and prelaunch checkout 
are discussed together with the significant 
incidents at each location. The acceptance 
rationale and the tank launch configuration 
are presented. 
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1.0 TANK MANUFACTURING AT BEECH AIRCRAFT 


1.1 GENERAL 


A detailed review was made of tank fabrication of oxygen tank 2 
S/N 1002^XTA0008 at Beech Aircraft Company (fig. l-l) . Assembly and test 
records, material review dispositions, and failure reports were reviewed. 
Due to the particular significance of the fan, heater, and density prohe 
wiring, a special review and demonstration of assembly methods of these 
components was made at the plant at Boulder, Colorado. (Data bank file 
no . 125 ) 

The installation procedures for heater, fan, and probe wiring were 
found to be critical, with several areas where routing damage would not 
be visible on the assembled product. (While mating a fan with the heater 
probe, the demonstrator ab raided the fan wiring at the motor exit point. 
This area was not visible with the fan completely installed. ) 

Assembly records for this tank revealed no significant data except 
for a prevalence of fan motor wiring problems. These had apparently been 
corrected by improved procedures and inspection requirements prior to 
fabrication of the motors installed in tank XTA0008 for delivery. None 
of the testing or inspections conducted gave any positive indication of 
actual pre-shipment damage . 

The tank was accepted with a waiver for an out-of-specification heat 
leak rate. This appeared to have no bearing on the flight failure. 


1.2 MANUFACTURING HISTORY OF CRYOGENIC OXYGEN TANK 2 


Pres sure Hemispheres . - Rough machine forged pressure hemispheres for 
the Apollo 13 cryogenic oxygen tank 2 were received by Airite Corporation 
of Sargent Industry from Cameron Iron Works in April 1966. The properties 
such as physical, chemical and grain size were verified, vising radio- 
graphic inspection techniques. After the pressure vessel hemispheres were 
machined and welded, they were dimensionally inspected and X— rayed. The 
heater and quantity probe support brackets (provided by Beech Aircraft 
Company) were installed and the pressure vessel girth welds then accom- 
plished and X-rayed. The tank was shipped to Beech Aircraft Company in 
June 1966. The pressure vessel was proof tested at approximately 
1,325 psig and leak checked at approximately 925 psig. 
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Figure 1-1.- Manufacturing and test flow for the oxygen tank at Beech. 
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A total of seven insulation layers were installed on- the pressure 
vessel, with an inspection after each layer, and the vapor cooling supply 
tube installed between insulation layers three and four. This was accom- 
plished between 27 June and 5 July 1966. 

The outer shell assembly halves were deep-drawn and chemically milled 
at Chemtronics , Inc. After the physical and chemical properties were ver- 
ified, the halves were dimensionally inspected, welded, and X-rayed. Dur- 
ing X-ray of the lower assembly on 28 March 1966, a porous area was found 
around the vacuum valve . This was replaced hy material review action with 
an alternate part on 27 May 1966 per MRB C55I+61. The two outer shell 
halves were then installed over the insulated pressure vessel, welded and 
X-rayed. 

Quantity Probe .- The quantity probe was provided by- Simmonds to Beech 
Aircraft Company on June 20, 1966, with no open items. During the manu- 
facturing process, the support sleeve bottom assembly required material 
review action. Upon receipt from Simmonds, the probe was assembled into 
a welded assembly containing the pressure vessel plug, feed tubes, elec- 
trical conduit and filter body. This process was completed 2h June 1966. 

The motor, fan, and heater assembly originally installed in t ank 
XTA0008 was S/N F620230. • 

The fan heater probe was assembled by welding and brazing the in- 
dividual component parts. At the Beech Aircraft Company appropriate 
dimensional and X-ray inspections were made. The heater elements were 
silver brazed to the central tube; the area was then cleaned and the 
fans were installed using shims as required to position the impeller 
properly with respect to the nozzle . After the heater wires were sol- 
dered, an assembly inspection, and dielectric and insulation resistance 
tests were performed on both heater and motor circuits . The motors were 
then operated in liquid nitrogen, and motor operation was verified by 
run-down testing. 

The filter discs were installed on the filter body. A bubble check, 
leak checks , and proof pressure tests were then made and the polytetra- 
fluoroethylene adapters and fill tubes were installed. After the quantity 
and temperature sensor wiring was pulled through the conduit, the quantity 
probe was welded to the pressure vessel plug. The probe assembly was in- 
stalled in the pressure vessel (fig. 1-2) , and the fan motor and heater 
wires were pulled through the conduit. The probe assembly was then 
screwed into the pressure vessel, the flange was welded to the pressure 
vessel flange, and visual and X-ray inspections were completed. After 
the wires were soldered to the connector, the connector ass emb ly was 
welded to the end of the conduit. The coil housing, consisting of the 
vacuum pinch-off tube, rupture disc, and vac-ion pump bracket, was welded 
to the outer housing of the tank and was X-rayed. An insulator in the 
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Figure 1-2.- Cryogenic oxygen tank assembly. 
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vac-ion pump assembly was found to .be cracked due to the use of incorrect 
welding wire on 30 June. The insulator- was replaced in early July. Vac- 
uum acquisition was begun on July 25th. 

This assembly contained motors serial numbers B648011 and B648016 
(Globe S/N 0^45^157019 and 7024). Globe had experienced handling and 
fabrication problems with Ceroc-T (Teflon over ceramic) insulation on 
the 36 gage wire used for the stator winding. Handling in the stator 
winding process caused phase to phase shorts on a high percentage of 
two lots assembled by one individual. 

Since the motors in this tank were from the suspect lot, vacuum 
acquisition was stopped and a special test was run; 12 hours operation 
in gas and 12 hours operation in LN 2 - Motor currents were monitored and 
resistance measurements were made after each test phase. During this test 
the upper fan motor became noisy and drew excessive current. Post-test 
data indicated a phase to phase short circuit. 

The motor fan heater assembly was replaced with S/N F620194 con- 
taining motors S/N B648022 and B648032 (Globe 7C30 and 7C4l) . Both of 
these motors were wound after implementation of new process controls 
and dielectric tests to avoid the earlier problems . These motors were 
received in early September 1966 and were used to replace two rejected 
motors S/N 648013 and 648017 (Globe S/N's 7C21 and 7C25). 

The replacement of the probe in the tank was accomplished 
9 September, 1966, with the assembly of the density .probe and coil 
complete by 15 September. The second vacuum acquisition was started . 

28 Septemeber and completed 18 October 1966. The tank then moved to 
the test area for the end item acceptance test. 

Several minor discrepancies during assembly were accepted without 
rework. These included oversize (.230 instead of .190 max.) holes in 
the electrical connector support, an oversize rivet hole (.137 instead 
of .132 max. ) in the heater tube directly over the lower fan, and a 
small dimensional error in the clamp ring area of the pressure vessel. 

This was in an area not affecting the final machined part . 

The initial end item acceptance test on 21 October 1967 indicated 
a heat leakage at min DQ/DM of .809 lb /hr. The specification requirement 
was .715 lb /hr. This test was repeated 24-26 October with a leak rate 
of .757 lb /hr. 

Following this run, the pinch off tubes were cut off and the 
vacuum jacket reevacuated during the period from 29 December 1966 to 
31 January 1967 . A final acceptance test was run on 3 February 1967 
resulting in a heat leak rate of .815 lb /hr at min DQ/DM. A special 
test at 100° ‘F was run indicating an average flow rate of ,726 lb /hr. 
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Oxygen tank XTA0QQ8 was accepted by a waiver of the heat leak rate 
dated 19 April 1967 and was shipped to North American on 13 May 1967 . 


1.3 SUPPORTING DATA 


This section contains a reference listing of the pertinent documen- 
tation reviewed in -support of sections 1 . 2 . 


1.3.1 Discrepancies Not Reworked to Drawing 

Pressure vessel .- MR C10846 out -of- tolerance condition on forging. 
O.K. per MR. 

Outer shell .- MR C5546l weld porosity. MR removed vacuum valve and 
installed an alternate part . 

Quantity probe .- No MR action at Beech, 

Motor, fan, heater assembly .- MR B01l42 Insulation scraped 1-3/4 inch 
from motor 648017. Returned to vendor. 

MR B01l40 Motor 648013 failed after 4 hours operation. 

MR B01133 upper fan motor. Failed during screening test after in- 
stallation in tank. Returned to vendor. 

MR B01184 reworked tank to install new motor, fan heater assembly 
with new motors . 

MR BOO 659 oversize rive.t hole .132 should be .128 in heater tube 
assembly. O.K. per MR. 

Tank assembly .- MR C43101 cracks in insulator alongside weld bead. 
Replaced. 

MR C 78515 oversize .230 should be .l80 - .190 inch diameter. 

O.K. per MR. 

MR B500016 flow rate higher than specification. 

MR C21381 authorized standard repair parts for re evacuation of 
annulus . 

NAA/NASA spec deviation CSM 0044 accepted tank with leak rate' out 
of specification requirement. 
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1.3.2 Procedures 

Beech BP l4 346—1 rev C. End item acceptance on oxygen tank assembly. 

Data "bank file package 125- Tank assembly pictures of cut-away tank. 

1.3-3 Other Documents 

Data Bank file package 2 - End item data package oxygen tank S/3J 10024 
XTA0008. 

Data bank file package 6 - Oxygen tank fan motors failure analysis 
report number 14510 . 

Data bank file package 14 - Simmonds probe end item data package. 

Data bank file packages 27 and 28 - Globe data for S/N JC30 and 7C4l. 

Data bank file package 34 — Globe engineering report 929 

Data bank file package 56 — Failure reports against oxygen fan 
motors . 

Data bank file package 90 — Beech inspection record oxygen tank 
S/N 10024 XTA0008. 
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2.0 CRYOGENIC SHELF INSTALLATION AT 

north: American 


2 .1 GENERAL 


A detailed review was made of 0 2 tank no. 2, its installation onto 
the eryo shelf and into the spacecraft , and all testing experienced during 
the 25 month cycle at Forth American, Downey (fig. 2-1). This included 
all test checkout procedures (TCP’s), discrepancy reports (DR’s), waivers, 
significant data taken during the TCP's and the fluid and gas sample re- 
ports. With the exception of a shock experienced by the shelf during its 
removal from S/C 106, possibly not adequately highlighted for considera- 
tion in the final decision to launch, all activities appeared nominal. 

All checkout procedures appeared adequate for testing to the level of 
detail required. Details of this incident and the subsequent analysis 
will be found in Para. 2.3 and 2'. 4-. The tank gave no indication of dam- 
age prior to shipment to Kennedy Space Center. 


2.2 CRYOGENIC OXYGEN SHELF HISTORY AT HR, DOWNEY, CALIF. - 


The- assembly of oxygen shelf, P/F V3T- 45 4200-211, S/N 06362AAG3277 , 
containing oxygen tanks, serial no. 10024XTA0009 in the number one posi- 
tion and serial no. 10024XTA0008 in the number two position was started 
at a sub-assembly level January 15, 1967 . 

The shelf structure had 5 discrepancies recorded which went through 
normal material review action. Fo discrepancies were noted against the 
pneumatic sub— assembly buildup containing the flowmeters, pressure switches 
and sensors, reactant control, check, and relief valves. 

ihe electrical equipment installation had a number of discrepancies 
which were returned to print or were corrected by standard repair pro- 
cedures. Two splices were added by material review action where pig-tail 
wiring from components was too short to reach the required terminals ■ 

The heater control box also had a number of minor discrepancies which 
were returned to print and two material review actions ; one accepts a 
condition of stress marks indicated on terminals of a diode board, while 
the other authorized change out of damaged terminal boards . 

All discrepancies against the cryo fan control box were corrected 
to print configuration. 


PRECEDING- PAGE BLANK NOT fit, M in 












11 


These sub -assemblies were- utilized in the build up of the total eryo 
shelf assembly which was started l4 April 1967 . Daring initial build up 
several braze joints required re-heating to pass X-ray requirements. 
MR58134-1 was initiated on 3-11-68' to document an indentation on oxygen 
tank S/N 0008 .070 in. deep, 1.50 in. in diameter approximately ’6 in. 
from the top edge of the flange. This was cleared by the material review 
board on 3-11-68 for unrestricted usage. 

Following completion of the fabrication, OCP M-10l4-88l0A was run 
on the shelf during the period of 27 April to 13 May, 1968 . This test 
verified integrity with a proof pressure test of 1262 PSIG, leak tested 
the system at jb-5 PSIG and functionally verified all fluid, mechanical, 
and electrical portions of the system. 

Several DR's, A99881, A22062, A22061, A22064 and A22063, associated 
with the relief valves were reviewed. These were associated with condi- 
tions established to assure cleanliness of valve seats prior to relief 
valve functional testing and were corrected by procedural changes . Ho 
functional problems with relief valve operations were noted. One weld 
leak was repaired (ref. DR A22067). DR A4l6l4 documented apparent cor- 
rosion around the welded seam where the lower pinch off tube flange 
attaches to the lower hemisphere of the outer shell. The area was cleaned 
and accepted for unrestricted use. DBA 25245 documented a ding on the 
— y side of O 2 tank 2 approximately 1 in, dia. by .005 in. deep. It was 
accepted by M. R. action. 

The oxygen shelf was installed in S/C 106 on 4 June 1968 . An in- 
stalled sub-system test, OCP-P-1518, was conducted between 3 August and 
8 August, 1968 . This checkout included operation of reactant valves, 
heater controls , fan motors , and control and display and caution and 
warning functions. Interface plumbing was leak checked at 900 PSIA. 
Several minor discrepancies caused by procedural or ground support equip- 
ment problems were reviewed. Hone appear to be significant. 

During the fuel cell simulator checkout (MA0706-1510 ) l4 to 29 Aug. 
1968, to confirm readiness for fuel cell installation, the flowmeters on 
the oxygen shelf were checked out. Ho discrepancies were noted. 

In August, a requirement was established to modify the vac-ion pumps 
to preclude arcs due to corona effects at altitude. To accomplish this 
modification per EO 702138 the oxygen shelf was removed from S/C 106 . 

During this removal on 21 Oct. 1968 , a screw was not removed and a 
handling fixture failed, resulting in the shelf dropping back onto the 
mount . 

The procedure for removing the oxygen shelf; Ho. 9EH- 5000-310 7R2 
dated 27 May 1968 calls for the following basic steps . 
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1. Check that LO 2 tank protective covers are installed. 

2. Position the LO 2 tank installation adapter 9EH-1009 (fig. 2-2) 
on the LO 2 tank shelf and engage the two thumb screws in the outer edge 
of the shelf to secure the adapter to the shelf. 

3- Hoist the counter weighted sling, 9EH1275 , with a bridge crane 
after assuring that the counter weights are positioned as marked for the 
no load condition. 

4. Bolt the sling to the adapter at the attach plate with six bolts . 

5. Adjust the counter weights (fig. 2-2) to the position marked for 
center of gravity with load attached. 

6. Apply tension with bridge crane. 

7. Disengage all connections between service module and L0 2 tank 
shelf. 

8. Using bridge crane, raise tank shelf until it clears the service 
module structure . 

9. Using bridge crane, slide shelf out to trunnion fixture. 

According to the technicians and inspectors, all steps had proceded 
normally until step 9 3 when adapter 9EH1009 failed at a weld joint between 
two 3 inch 1/8 inch wall aluminum tubes (fig. 2-2 point D), allowing the 
shelf with a portion of the adapter attached, to drop back down onto the 
service module support angles . The distance of the drop was estimated at 
the time to be approximately 2 inches . 

Investigation at the time brought to light the following discrep- 
ancies : 


1. A 10-32 screw attaching tank no. 1 and the shelf assembly to a 
bracket between radial beams 3' and 4 had not been removed, leading to 
high moments on the 9EH1009 adapter. 

2. The normal periodic proof load was 2 days overdue. This period 
was extended by NR and NASA QC for the emergency usage on this removal. 

3. The normal load and proof load, 200 lb and 300 lb respectively, 
were low for the actual estimated shelf weight of 247 lbs. 

4. The weld joint which failed had poor penetration and a lack of 
wetting with the cross section less than parent metal thickness in the 
tension area of the weld. 



To hoist 



Figure 2-2.- Oxygen shelf and, hoist assembly. 
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A re-view of the shelf ana the service module supporting angles was 
made at that time to determine if any damage had been incurred. A ding 
in the lower face sheet of the fuel cell shelf was noted. The location 
of the ding coincides within 1 to 2 inches of a potential impact point 
of the pinch off tube cover of oxygen tank number two. This is docu- 
mented on DR A80327 initiated 10-21-68, on- DR A112436 dated 10-21-68 and 
on DR Al40539» initiated 11-12-68 when the tank shelf was being prepared 
for retest after the modification. 

No discrepancies were noted by inspection and the retest require- 
ments to assure pressure vessel and system integrity were established 
on DR A 1^0539 with RASPO concurrance. 

A corrective action request (CAR) ( 80327 ) was initiated to identify 
corrective action requirements which included the following items : 

1. The procedure 9EH-5000-3107 was revised to specifically call 
out all structural attachments (ref. Rev. 3 dated Oct, 29, 1968). 

2. The handling fixture was redesigned to add gussets to the criti- 
cal joints. The proof load requirement was increased to 1*50 lbs with a 
rated load of -300 lbs. 

3. Welds were X-rayed. 

Subsequent to the Apollo 13 tank failure, a more detailed analysis 
(ref. para. 2.3) was made to estimate the shock loads imposed on oxygen 
tank no-. 2 and to assess the potential for hidden damage inside the tank. 

Following the modification of the vac-ion pumps and re-identifi cation 
of the oxygen shelf to a -71, accomplished between X Nov. and 17 Nov. , 
1968 , the shelf was retested to verify integrity as well as to verify 
the modifications. 

This testing included proof pressures to 1276 PSI, external leakage 
tests and functional tests of pressure transducers and switches , thermal 
switches, and vac-ion pumps. Sequences not performed during the retest 
included: 


A. Seq. 2.3 Fan motor functional check 

2.k and 2.5 Signal conditioner checks, density and temperature 

2.11 Coupling leak check 

2.12 Check valve internal leak check 

2.13 Fuel cell reactant valve functional and leak check 

2 .14 Flowmeter functional test 
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These parameters were verified during the installed systems tests 
in building 290. 

The only significant DR was DR Al424l2 which indicated possible oil 
contamination from facility lines . The GSE fill hoses were checked and 
found not to be contaminated, indicating that potential contaminants had 
not reached the S/C interfaces. This was further verified by the vent 
line samples taken during servicing operations at KSC. A review of the 
shelf testing showed that the only portions of the shelf test not re- 
peated at this time were not associated with tank or system integrity or 
the vac- ion pump rework . 

DR's 104283 and 104284 indicated an out of specification condition 
on the heater thermal switches. Review showed that, since the test was 
for functional verification only, stabilization periods were not required, 
resulting in temperature differences between the resistance element and 
the switches . 

following completion of the shelf retest, the shelf was installed 
in S/C 109 on Nov. 22, 1968. 

During the installation, one line required rework to eliminate a 
structural interference and a tube joint on the line failed X-ray and 
required reheat . 

Retest of the cryogenic shelf installed in S/C 109 per OCP P-1518 
was accomplished during the first two weeks of January 1969* This test 
verified operation of shelf and tank functions through spacecraft con- 
trols . 

A special retest of the tank pressure gages was run at the request 
of engineering to re-verify panel meter readings. 

No significant anomalies were noted. Following completion of fuel 
cell simulator testing and overall testing of the spacecraft, S/C 109 
was prepared for delivery to KSC, following a normal closeout inspection. 

During this period, DR A150485 documented a .005 inch deep ding in 
a line to O2 tank 2. This was checked for cracks with dye penetrant and 
was polished out. The spacecraft was shipped to KSC June 25, 1969- 
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2.3 SHELF DROP INCIDENT EVALUATION 


The following paragraphs estimate the most probable sequence of 
events, investigate the dynamics, describe the mathematical model used, 
make comparisons with the tank qualification testing, and investigate 
the tank internal structure . It is concluded that the internal compon- 
ents should be adequate to withstand axial acceleration levels in excess 
of 100 g, and that the- tank should not have experienced more than 12.7 g 
with a predominate frequency of 27-8 Hz. 

The qualification testing did not provide a positive method of de- 
termining the damping between the tank and its internal components , how- 
ever , the structural review revealed no critical items. 


2.3-1 Handling Sequence 

Since the records do not positively define the exact sequence of 
events, several sequences were postulated within the limits controlled 
by the available strength and the mass properties of the fixture. 

Photographs of the failed weld do confirm that failure initiated 
in the weld at the top of the joint; such a failure is consistent with 
a bending moment which induces tension in the top fiber. An estimate 
of the strength of the section at the weld, based upon dimensions scaled 
from the photograph, yielded a mi nim um failing load of 8500 inch-lbs 
moment with an upper bound of 1 6 000 inch-lbs. The minimum value gives 
a positive margin of Vo% (to failure) for the one-g balanced position of 
the assembly. Investigation of the mass properties of the handling fix- 
ture determined that the center of gravity shift was limited to approxi- 
mately 10 inches (maximum possible counterweight movement). 

Possible sequences are illustrated and numerically described in 
Figure 2-3. Case 1 postulated that the crane was raised excessively and 
thus over loaded the shelf assembly through contact with the fuel cell 
shelf. This sequence was excluded, however, because this loading reduces 
the bending moment from its initial value and tends to relieve the bear- 
ing pad forces which interface with the bottom of the shelf. If this 
performance were undetected and the forces were increased until failure, 
the failure would result from moment which induces tension on the bottom 
fiber rather than the top fiber as deduced from examination of the photo- 
graphs . 

Case 2 postulates that the crushing of the core of the fuel cell shelf 
acts as a constant load device and that the counterweight center of gravity 
is shifted in an attempt to level the oxygen shelf. This sequence requires 
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1465 lb 



Load representations 



Case 1 - Fuel cell shelf contact 

Assume a force, exerted by fuel cell shelf. 

For an X„>406 lb, the ground support equipment 

rt 

bearing pads swing free: This loading condition 
reduces the moment at the failure point to 15 percent 
of its balanced one g value, i.e., M = 1110 in-lb. 

Conclusion: Sequence excluded 


1591+AP 


300 lb 



Case 2 - Counter weight shift and shelf contact 

Assume shelf contact force to be constant 300 lb 
and counter weight C.G. adjusted Ad to balance loads 
Moment, M, at failure point: 8500<M<16,000 in— lb 


Note: Sequence requires 
large shift of counterweight. 

Maximum possible shift = 10 inches. 


M, in-lb 

R, Jb 

AP, lb 

Ad, in 

8500 

70.3 

103.7 

7.9 

10,000 

43 

131 

10.0 


Conclusion: Sequence unlikely 


1465+AP 



Case 3 - Counter weight shift 

Moment, M, at failure point: 8500<M<16, 000 in-lb 
Counterweight C.G. adjusted to balance loads: 
Maximum possible shift = 10 inches. 


M, m-lb 

R, lb 

AP, lb 

Ad, in 

8500 

-22 

22 

1.7 

14,500 

-131 

131 

10.0 


Conclusion: Most probable sequence 


Figure 2-3.- Shelf handling sequences. 
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■very large movement of the counterweight center of gravity to produce mo- 
ments consistent with the failure . For example , a shift of 10 inches in 
center of gravity requires that the counter weights he moved to the out- 
hoard stops; moving one counterweight only to achieve this configuration 
requires' a shift of 29 inches. This sequence is considered unlikely. 

Case 3 postulates adjustment of the counterweight center of gravity 
to attempt to level the assembly. As noted, this procedure requires less 
than a 2 inch shift of the counterweight center of gravity to achieve 
failure of the fixture . This sequence agrees with most of the witness 
statements and is considered the most probable. 


2.3.2 Dynamics 

Because of the ding in the underside of the fuel cell shelf, the 
dynamics of the oxygen tank shelf were considered. 

Case I .- The rate at which the weld fails, as shown in figure. 2-4, 
determines the displacement and velocity of the top of the tank. Calcu- 
lated times greater than 0.1 second were required to produce an, impact 
of the tank and the fuel cell shelf. This slow failure of the weld was 
considered improbable. 

Case II .- A one-half scale layout of the configuration illustrated 
a potential mechanism for contact. The weld in question joins cylindri- 
cal sections and thus may constrain the counterweight fixture until the 
angular displacement allows complete separation. This was assumed and 
displacement calculations were performed 1 assuming zero moment at the 
failed joint. Such an assumption is conservative and yields velocities 
at contact of 20 inches /second to 35 inches /second depending upon the 
shelf position at the time of failure. 

Based upon the foregoing, calculations were performed to predict 
the transient dynamic response of the tank due to impact upon its sup- 
ports. These were then adjusted to account for the strain energy stored 
in the fuel cell shelf and oxygen tank shelf for an impact which pro- 
duces the observed ding. Hand calculations were then performed to esti- 
mate the response of the tank at the time of impact with the fuel cell 
shelf. 


2.3.3 Mathematical Representation 

A three dimensional, finite element mathematical representation of 
the 0£ shelf assembly was generated to predict the transient loads sus- 
tained. The model included structural idealizations of the bonded honey- 
comb shelf assembly, the number 1 and 2 tank assemblies, and that portion 
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of GSE that remained attached to the shelf after the fixture failed. The 
sandwich portion of the shelf structural assembly was idealized hy plate 
elements, which react loadings normal to the surface, with the chemically 
milled regions included hy reducing the plate tending stiffness. The 
close-out members along the edges of the shelf and the tank cut-outs were 
idealized as team elements. The handling fixture that remained with the 
shelf was modeled as a system of beam, bar, and torque tube elements. 

The radial beam stiffeners act as supports for the shelf and were repre- 
sented by springs to ground. 

The analytical description of the tank assemblies consisted of the 
inner and outer shells represented by generalized plate elements which 
provide both membrane, and plate stiffness. The load path between the 
inner and outer tank is unique in that the loads must be reacted by com- 
pression of the insulation material. To simulate this load path, the 
inner shell was attached to the outer shell in the lower hemisphere by 
means of links which react load normal to the shell. The complete struc- 
tural idealization of the shelf assembly consisted of approximately 
1400 elastic degrees of freedom. 

Inertia loadings were assumed significant in the vertical axis only, 
and therefore only these terms were retained in the response analysis. 
Shelf structural weight and tank weights were appropriately distributed 
to respective displacement points. Weights of valves and electrical 
equipment on the shelf were assumed rigidly attached to the shelf. The 
weight of the tank insulation was distributed on the inner shell in the 
upper hemisphere and on the outer shell in the lower hemisphere . The 
quantity gaging probe and heater assemblies were assumed rigid and their 
weights were reacted at the top of the inner shell. Approximately 290 dy- 
namic degrees of -freedom were retained for the dynamic response calcula- 
tions . 


2.3.4 Analysis of Results 

Three independent sets of assumptions and resulting initial condi- 
tions were specified and analyzed. 

For the first case, the oxygen shelf was raised so that the pinch off 
tube cover was in contact with the fuel cell shelf and the oxygen shelf 
was allowed to free fall. The initial internal strain energy in the oxy- 
gen shelf was conservatively estimated and assumed to increase the veloc- 
ity of the shelf at contact with the radial beam stiffeners . Results 
indicate a peak acceleration of 11.8 g's on the quantity gaging probe 
ana heater assembly. Predominate frequency content was 27-8 hz, the 
fundamental frequency of the 0 2 shelf in contact with the radial beam 
stiffeners . 
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The second case considered the transient response due to impact of 
the tank and the fuel cell shelf. Conservative estimates were made of 
the fuel cell shelf stiffness; relief due to crushing of the honeycomb 
was neglected. An impact velocity of 35 in/sec was assumed; such a veloc- 
ity is conservative since it assumes an initial clearance of 2.5 inches 
(the shelf had not been lifted) and a pinned joint at the location of the 
failed weld. These assumptions and initial conditions yielded a peak 
response of the heater /probe assembly of 19*1 g with a predominate fre- 
quency of 33.6 hz. 

MSC test ( TPS-013-T-055 ) was run using a representative fuel cell 
shelf and vacuum pinch-off tube cap. For levels in excess of 7g, perman- 
ent deformation could be observed in the tank dome cover, thus limiting 
the upward shock experienced by oxygen tank 2 . 

For the third case, the strain energy stored in both the oxygen 
shelf and the fuel '0611 shelf was conservatively estimated. This energy 
was then assumed to increase the velocity at contact with the radial beam 
stiffeners. This condition resulted in 12.7 g. As in the first case, 
the predominant frequency was 27*8 hz. 

2.3.5 Comparison with Quality Test Environment 

The qualification test reports of the 0 2 tank were reviewed to de- 
termine if the tank design had been previously exposed to acceleration 
levels comparable to those resulting from the shelf drop. Testing con- 
sisted of random vibration tests to simulate flight random environments , 
and sinusoidal sweeps from 10 to 2000 hz with approximately 1 .0 g zero to 
peak input. The sweep rate was 1.0 octave- per minute. 

Accelerometers were not located within the tank during these tests 
so it is not possible to determine conclusively if the quantity gaging 
probe and heater assembly experienced these levels. The transient re- 
sponses that occurred during the incident were primarily at a frequency 
of 27.8 hz — the natural frequency of the shelf assembly resting on the 
radial beam stiffeners. 

Responses during the sinusoidal sweep testing are similar in nature 
to those predicted by the foregoing analysis, however, an accurate estima- 
tion of the damping of the tank assembly is required to predict the in- 
ternal response levels achieved during the sinusoidal sweep tests. ®o 
data is available to estimate the damping. 
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2.3.6 Tank Structural Review 

The structural assembly of the tank was reviewed to identify any 
vibration sensitive components. Particular attention was focused upon 
the quantity probe and heater assemblies. Structural analyses were per- 
formed of these assemblies. All components were found to be adequate 
for axial g levels in excess of 100 g. 

Because of the detanking problems at KSC (ref. paras. 3.1.1 and 3.3) 
and the postulations concerning the condition of the fill tube plumbing 
at its interface with the quantity prob.e standpipe, the loads on the tube 
were reviewed. This assembly weighs approximately .03 pounds; at ±20 g 
the force exerted is ±0.6 of a pound. Conservative estimates of fluid 
loads (ref. para. 3-3. 6. 1.3) during filling or detanking are in excess 
of such forces by at least one order of magnitude. It is concluded, there- 
fore, that the transient loading experienced by this tubing was consider- 
ably less severe than that expected during normal service. 


2.4 SUPPORTING DATA 


This section contains a reference listing of the pertinent documen- 
tation reviewed in support of sections 2.1 and 2.2. 


2.4.1 Discrepancies Not Reworked to Drawing 


Electrical Equipment Installation .- 


DR A44409 
A44410 
A58128 
A58129 
A58130 
A5 8136 
A71395 
A71398 


Wires too short. Corrected by MR splices. 


Split grommets. Corrected by std. repair EL 1.1. 
Insulation damage corrected by std- repair EL 1.1, 


Shelf-Oxygen Tank .- 


DR A31572 
A414071 
A392227 
A428326 
A435167 


.135 gap between -7 angles. 0k per MR. 
Void. Rework per MR. 

Void. 0k per MR. 

Doubler interference. Reworked per MR. 
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Heater Control Box .- 

DR A448021 Stress marks on terminals of diode board. Ok per MR. 

A18022 Defective part replaced instead of reworked. Ok per MR. 

Shelf Buildup . - 

410009 

410038 

410039 

409959 Braze joints failed X-ray. Corrected by reheating. 

A17874 

AI 788 I 

A001181 

A58134 Dent in outer shell tank 0008 .070 in deep x 1.50 in. 

0k per MR. 

Shelf Test for S/C 106 OOP 10l4 .~ 

A25245 Ding on - 4 side tank 0008 1 in dia. 0k per MR. 

A22057 Recessed pin on connector. Corrected by DR action. 

A22063 Plow rate during purge of relief valve low requirement 

changed. 

A4l6l4 Suspect corrosion around tank weld seam. Cleaned and 

acceptable . 

A55909 Sect. 2.15.22 not performed due to lack of Rayco seals. 

Seals replaced and checked during later testing. 

Installation and Test in S/C 106.- 


DR Ho. 1 Insulation cracked on yellow wire ,S48TK4P1. Damaged' 

area wrapped with Mystic tape. 

DRA102717 Additional documentation of Rayco seal problem. See 

DR A55909. 

Shelf Removal from S/C 10 6 .- 

A80327 Documents the shelf dropping incident during removal from 

S/C 106 and the inspection requirements . 

A112436 Documents a ding in the under side of the fuel cell shelf 

on S/C 106 .021 in deep. 

A140539 Documents the inspection for damage and retest require- 

ments for the oxygen shelf. 
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Tank Rework for Vac-ion Pump Modification . - 

A106855 Connector locking feature improper. Be worked by Cannon. 

A104285 Line to fuel cell bent. Leak check performed. 

A102975 How transducer connection leaking - reheated. 


Shelf Retest. - 


A104283,4 

A104286 

A106866 

A104287 


Heater switches out of tolerance. Insufficient stabiliza- 
tion time allowed. 

Scratch on line. Polished out by MR. 

Bent line MR /re inspect . 

Shelf mowed prior to part test review. 


S/C 109 Retest.- 


A9U287 

A55700 

A137385 

AI50U85 


Tube rides 
Tube joint 
Line leak. 
Line ding , 


against structure. Reworked tube, 
f ai le d . X-ray . Reheat e d .. 

Seal replaced, 
polished out by MR. 


2.4.2 Test Procedures and Specifications 


MAO 703-1014 OCP M-1014-8810A - Shelf proof pressure leak check and 
functional checkout . 

MAO 703-1016 Service module tubing proof pressure and leak test. 

MAO 703 1518 OCP P 1518 - Cryogenic storage system installed system 
checkout . 


MAO 706-1510 Fuel cell simulator checkout. 

9EH-5000-3107R2 27 May 1968 Cryogenic shelf installation and removal 
9EH-5000-3107R3 29 May 1968 procedure . 


2.4.3 Other Documentation 


SSAD Books 

Section 3.19 These books contain copies of as run procedures, DR’s, 
MR's, waivers, and fluid and gas sample reports. 

Data Bank File No. l49 

Apollo 13 Investigation Report, 

Oxygen shelf incident, 

Vol. I and II 


Contains photographs, procedures , 
etc., pertaining to the drop. 
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3.0 CRYOGENIC SHELF CHECKOUT AT KENNEDY SPACE CENTER 

3.1 GENERAL 


A detailed review was made of the behavior of oxygen tank 2 during 
its 9-1/2 month checkout cycle at the Kennedy Space Center. This included 
all Test Checkout Procedures (TCP’s), discrepancy reports (DR’s.), waivers, 
significant data taken during the TCP's, and the fluid and gas sample 
analysis reports. With the exception of the inability to detank liquid 
oxygen in the normal manner during countdown demonstration test (CDDT), 
and the subsequent procedures utilized, all operations were nominal. All 
checkout procedures appeared adequate for testing to the level of detail 
required. 

In addition to the historical summary, four specific areas: detank- 

ing, fan performance, quantity gaging checkout, and system purity, are 
discussed below. The tank gave no indication of pertinent damage prior 
to launch. 


3.1.1 Inability to Detank 

A review was made of all available facts pertinent to this incident; 
the history, the data taken, the theories developed, the technical and 
management personnel involved, and the rationale utilized to accept the 
tank for launch. This is presented extensively in section 3-3 together 
with the post flight analysis and testing conducted. 

In general, liquid oxygen tank 2 could not be off loaded after its 
initial filling during countdown demonstration test . The problem was as- 
sumed to be due to missing or broken pieces, or to a tolerance buildup 
under cryogenic conditions, within the dog leg portion of the tank fill 
path (reference Figure 3-1 )- 

The effect of this damage on the tank in-flight performance was 
judged to be inconsequential since all other aspects of tank performance 
were nominal. In addition,- the available energy was determined to be 
minimal both from small Teflon particle fan impeller impacting and from 
probe plate -to -pi ate shorting by the inconel tube. 

After numerous attempts with gaseous oxygen purges and higher expul- 
sion pressures, the fluid was boiled off through the use of tank heaters 
and fans, assisted with pressure cycling. As established from the re- 
corded data, the heater thermal switches did not open. There was no 
evidence of concern or consideration given to the possibility of internal ‘ 
wire damage resulting from overheat. There are no apparent specifications 
that prohibit this activity. 
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The 2TV1 detanking problem, significant in the acceptance of the 
fill path gap theory, proved, upon close review to be only a difference 
in the time required to empty each tank, with both times being consid- 
ered nominal. (Reference section 3.^) 

During post launch testing (Ref. section 3.3.6) it was found that 
with ground power (65 V dc and 6 amps), the heater thermal switch con- 
tacts will weld closed in the initial opening attempt with currents as 
low as 2 amps . Continued application of heater power will produce tem- 
peratures of 900° F to 1000° F in the area where the lower fan wiring 
routes behind the heater element . Severe insulation damage results . In 
addition, fan motor case temperatures will reach 350° F. The type of 
functional tests performed after the detanking would not necessarily de- 
tect these conditions. 


3.1.2 0 2 Tank Fan Motor Performance 

Fan motor performance was evaluated continuously at Beech Aircraft, 
at North Ameri can-Down ey , and at KSC. Prior to installation in the space- 
craft, the voltage generated by the inertial coast do™ of each motor is 
displayed on an oscilloscope, the pattern examined, and the rundown time 
measured. After spacecraft installation, due to the service module wir- 
ing configuration, oscillograph traces are made of AC phase B while both 
fan motors are rundown in parallel. These are examined for pattern only. 

All performance data pertinent to oxygen tank 2 (SN XTA 000 8 ) was 
examined and compared with oxygen tank 1 (XTA 0009) and previous data 
available from ‘S/C 106 and 108. No anomalies were detectable. Of 13 
oscillograph traces examined, 7 showed both motors in synchronization 
during coast while the remaining 6 showed similar irregular nodal patterns 
produced by summation of the frequencies from each motor. Both types of 
patterns are considered normal since generators in parallel tend to feed 
energy to each other and remain in synchronization. All fan data examined 
was essentially identical. 

Test records were reviewed during all fan motor operations at KSC 
for any indications of unusual AC bus transients. Thirty periods of 
time were found when oxygen tank 2 fan were powered. No transients were 
found except those normally associated with fan motor turn-on and turn- 
off. Fan motor performance was considered normal. 
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3.1.3 0£ Quantity Gaging System Checkout 

An extensive data review of the behavior of the O 2 quantity gaging 
system was conducted. The system was powered in five separate tests 
over the 9-1/2 months period at KSC for a total of 167 hours, 8 minutes, 
and coincident with 28 fan on/off cycles over the 17 day period of CDDT 
and launch count . 

Throughout these tests, the gaging system in tank no. 2 exhibited 
less sensitivity to noise and transients than did that in tank no. 1 . 
Uormal transients of from 1-4 percent were visible with most of the fan 
switching. 

No abnormal behavior of the quantity gaging system was evident prior 
to launch. 


3.1.4 O 2 System Purity 

The procedures used for liquid and gas servicing were examined to- 
gether with the laboratory sample reports used to certify purity both in 
the test complex and in the fluid media being introduced. A summary of 
these procedures along with the sarnie results can be found in paragraph 

3.4.2. 

A basic assumption is made that significant impurities (helium, 
nitrogen hydrocarbons ) will be detectable in the liquid overflow from 
the tank, and that the particulate size can be adequately controlled by 
the extensively qualified Wintec filter in the inlet line. Further 
assurance is obtained by maintaining the security of each test complex 
with a gaseous pad pressure and by introducing only sampled, certified 
media. 

All procedures appear to have been followed with no waivers required. 
In addition, the inlet line filter was examined by the vendor Wintec. 

The bubble point correlated to an equivalent pore diameter size of 9-4 
microns , with no evidence of contamination detrimental to cryo system 
operation. 
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3.2 OXYGEN SHELF HISTORY AT KSC 


The service module is received at KSC (fig. 3-2) with a pad pres- 
sure of 3-5 psi nitrogen. This remains until the initial leak checks 
are accomplished. During periods "between tank tests , a pad pressure 
using helium is maintained. Pressures used are approximately 3 psi in 
the altitude chamber and until first testing at Pad 39 , and then 80 psia 
thereafter. Oxygen is retained between countdown demonstration and launch 
count. 


The following paragraphs delineate the specific testing involving 
oxygen tank Ho. 2 and adjacent pertinent areas. 

TCP 3063 - Service Module Move and Mate Preps .- Sector IV panel 
(SM-2) was removed and the A34-380 pressure” operated disconnect (POD) 
support bracket and cryo tank No. 2 protective cover were installed. 

The fuel cell/cryo system receiving inspection disclosed no significant 
discrepancies . 

TCP3071 - CSM Mate .- Oxygen tank No. 1, and plumbing as sociated 
with the command module ECS system, were pressurized to 900 psi (fig. 3-3) 
with helium for 1 hour and 20 minutes to leak check the CM/SM interface 
connections. No discrepancies were noted associated with either the oxy- 
gen system or the prior facility validation TPS (GS-Q88-1-473) . 

TCP 0070 - Abbreviated Combined Systems Test , July 15 , 19 69 to 
July 17 , 1969 . - A leak check of the pneumatic operated disconnects (POD) 
was run using helium at 108 psia in tank No. 1 and 94 psia in tank No. 2 
(fig. 3-3). 


During fuel cell leak and functional tests, tank No. 2 was pres- 
surized to 1025 psia, establishing relief valve cracking pressure, de- 
creased to below 870 psia, and then increased to 954 psia to verify 
pressure switch operation. 

The vac-ion pump verification tests for oxygen tank No. 1 were in- 
conclusive and the test was rerun. The DC-DC converter was found to be 
faulty (IDR 0l4 ) and was replaced. Retest was accomplished during the 
spacecraft power up for TCP 0048. No other discrepancies were noted on 
the oxygen shelf. Hydrogen tank No. 2 pressure transducer failed 
(IDR 033). The defective transducer remained installed in the space- 
craft and a replacement transducer was installed in the H 2 tank No. 2 
supply line, under the H 2 shelf assembly, per TPS-S/C-109-SC042 . 

Retest was accomplished per TPS-SC-109-SC045 during TCP-0048, simulated 
altitude test. 




Figure 3-2.- Test flow for the oxygen shelf at KSC. 


























J Gaseous 
nitrogen 
facility 
1(3000 ps i) 

Nitrogen 
supply for 
validation 
sampling 


Gaseous 
helium 
facility 
(6000 psi) 


Line 226 


Facility valve 


1 

1 

Pressure 

regulator 

unit 

i — L 


i 

— 


Helium supply for 
system pressure 
checks 


Gaseous oxygen supply for simulated 
altitude and altitude chamber tests 


Figure 3-3.- Oxygen servicing facility (MSOB). 





32 


DR 109 - S/C 15k - First Fuel Cell Change - July 18, 19^9, to 
August 7, 1969 .- All fuel cells were replaced to install block 1 water 
glycol bypass valves. Block II valve operation inflight had disclosed 
that the valve was sensitive to particle contamination generated in the 
system. The post installation fuel cell interface leak checks did not 
pressurize the cryo tanks. Precaution, such as use of protective covers, 
was used to avoid damage to cryo tankage during this work and no discrep- 
ancy reports were initiated. 

TCP 00^8 - Simulated Altitude (Chamber Test) - August 13 - 
August 29, 1969 .- Oxygen for the environmental control system was 
furnished t hr ough port OP. The gas flow did not pressurize the tanks 
(fig. 3-3). 

TCP 003^ - Altitude Chamber Test - September 5 to September l6, J 1969.- 
GSE was used to supply oxygen to the environmental control systems through 
oxygen purge port (OP) for three altitude runs on 9-5 (unmanned), 9-9 and 
9-l6 (fig. 3-3). During these runs, tank No. 1 was pressurized to 9 
95k, and 930 psia but tank No. 2 was isolated. No anomalies were encount- 
ered during this operation. 

TPS SC-109-SC-118, SC-119 and DR SC-109 - SC0325 - October 9, 1 969 
to December 1969 .- During the launch count of SC 108, hydrogen tank 
No. 2 failed when loaded with liquid hydrogen. The failure was due to 
a leak in a bi-metal joint at the top of the tank . A decision to re- 
place this tank with hydrogen tank No. 2 from S/C 109 initiated the 
removal of SM-2 door from S/C 109 per TPS SC 109 - SC 118 and the re- 
moval of the hydrogen tank No. 2 per TPS SC 109 - SC 119. A later de- 
cision to replace S/C 109 hydrogen tank No. 1 resulted in again removing 
SM-2, replacing the tank, and reinstalling SM-2 per DR SC 109 - SC 0325- 
During this task no discrepancies were noted involving the oxygen tanks . 

TCP 0005 ~ Integrated Test With launch Vehicle Simulator - 
January 5/ 1970 to January 9, 1970 •- In preparation for fuel cell 
activation, the cryo tanks were pressurized with helium to 68 psia for 
a leak check of the POD's. The cryo tanks were then evacuated to less 
than 5 mm for 2 hours, followed hy pressurization to approximately 
80 psia with hydrogen and oxygen gas. The tanks remained at this pres- 
sure during fuel cell pressurization since the cells were supplied gas 
from GSE through the oxygen purge port OP (fig. 3-k) . The instrumenta- 
tion was verified, and fan motor rundown tests verified both motors were 
operating for all cryo tanks. 

During the fuel cell test, fuel cell No. 1 hydrogen regulated 
pressure transducer failed (DR SC 109-SC 0366). The transducer was 
replaced subsequent to fuel cell shutdown. It was also observed that 
fuel cell No. 1 load sharing was at a seven ampere variance with fuel 
cells No. 2 and 3. In addition, the current from fuel ‘cell No. 1 was 




Figure 3-4.- Oxygen servicing facility, launch complex 39. 
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fluctuating approximately 7 amperes in a random cyclic fashion when 
under a constant load (DR S/C 109-SC0387) • The problem was isolated to 
fuel cell No. 1. During the above, no anomalies were noted in the oxy- 
gen system, the tanks, or the associated test complex. 

DR SC-1Q9-SC 380 January 20 to January 29, 1970 Fuel cell No. 1 
was replaced due to the anomaly noted. During its removal and replacement, 
DR's SC 109-SC 0373 (incorrectly inserted pin in connector F62) and 
SC 109-SC 0374 (a shiny first thread on connector J1 which indicated the 
anodizing had been removed) were initiated against fuel cell No. 2. 

P62 was repaired by inserting the pin further into the lock position, with 
J1 accepted "as -is" for flight. No anomalies were noted for the oxygen 
shelf during the fuel cell 1 replacement. 

TCP 0028 - Flight Readiness Test - January 2 6 to February 2, 1970 
-Operations during this test were similar to those for TCP 0005 except 
that the fan motor checks were not planned. POD leak checks were re- 
quired to validate the installation after SM 2 door removal for fuel 
cell ehangeout. Vacuum and pressure cycles were the same as for TCP 0005 . 
The evacuation was followed with tank pressurization to approximately 
80 psia with oxygen and hydrogen gas . All three fuel cells were act- 
ivated and all operated normally. 

TCP 0028 - Flight Readiness Test Re-Run - February 18-27, 1970 .- 
ECS oxygen was supplied through the purge port without pressurizing the 
oxygen tanks . 

No anomalies were noted for the oxygen shelf during this activity. 

TCP 5127 (EPS Water Glycol Servicing and Compressibility) and 
DR SC 109-SC 044l. Februaiy 23 to March 8, 1970 .- This accomplished a 
volume exchange of the EPS Water Glycol system within the required 
45 days prior to launch. During the volume exchange a pressure increase 
was not observed when fuel cell No. 2 water glycol pump was turned on. 
Subsequent tests showed that the pump exhibited slow starting characteris- 
tics (DR SC 109-SC 044l). Due to this, fuel ceils No. 2 and 3 were re- 
placed. It was necessary to remove fuel cell No. 3 to have access to 
No. 2 in the rear of the SM bay. Interface leak checks of the fuel cells 
were conducted by pressurizing through the oxygen and hydrogen purge port. 

TCP 0007 - Countdown Demonstration - March 16 to March 19, 1970 .- 
P0D leak checks were again performed with 80 psia gaseous helium to 
validate the final installation after SM 2 removal. Fuel cells No. 2 and 
No. 3 were activated, resulting in a vacuum cycle of the cryo tanks, fol- 
lowed by pressurization to approximately 80 psia with oxygen and hydrogen. 
Moisture samples were taken from the tanks and verified to be less than 
2.5 ppm (oxygen tanks 1 and 2 read less than 2 ppm). This was accomplished 
by pressurizing the tanks with oxygen and hydrogen gas to 80 psia through 
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the vent line , venting the tanks hack through the vent lines and then 
obtaining a moisture sample at the vent line sample valve . The fuel cell 
activation and deactivation was completed successfully in the precount. 

During midcount, the spacecraft was serviced with liquid hydrogen 
and liquid oxygen (fig 3-5) • In preparation for this the portable dewars 
had been serviced, (fig. 3-6) sampled, and delivered to the test complex 
(reference section 3*1-^ - Eastern Purity). Immediately prior to hydrogen 
servicing all quantity prohes were activated. During hydrogen loading 
the quantity measurement SC 0032Q for oxygen tank number one showed ran- 
dom fluctuations of approximately ten percent. -This occurred for 22 min- 
utes duration (DR SC 109-DR 0521). A detail review was conducted but no 
cause -for this condition could be found. The sample rate of once per 
second for the measurement made any correlation to noise difficult. The 
quantity measurement for hydrogen tank number one, and the temperature 
measurements -for hydrogen tank number one and two, powered through the 
same .circuit breaker, did not experience the problem. This measurement 
remained slightly noisy, but functional throughout the. remainder .of- the 
testing , and was accepted for launch . 

Immediately after hydrogen servicing, the LH 2 GSE was secured, and 
the hookup -of the -LO^ dewar and purging of the oxygen fill system per- 
formed. Oxygen was then serviced (reference section 3.^.2). Plow rate' 
during oxygen servicing was about 25 pounds per minute (both tanks filled 
; in parallel) with a dewar pressure of approximately h5 psia. During ser- 
vicing the tank pressures read approximately 32 psia. Oxygen loading was 
normal although during the initial line purges a regulator which pressures 
the dewar, failed closed and had to he replaced. Heater checks after ser- 
vicing were completed with no abnormalities . The oxygen tanks were pres- 
surized to 331 psia during the heater test. All purity samples taken 
during loading, both from the dewar and the tank vent system, met specifi- 
cation (reference section 3.U.2.1). 

Following these operations, hydrogen offloading and a detanking to 
approximately 60 percent for the oxygen system was planned. (Oxygen tanks 
are normally left partially filled to support the ECS system during wet 
and dry CDDT. ) Hydrogen deservicing was satisfactorily completed. During 
the oxygen quantity adjust, oxygen tank 2 did not detank normally. A sep- 
arate summary of this detanking problem and the related operations will be 
found in section 3.3. The problem was documented on DR GS 132-1-1735 
(IDR 023) and DR SC 109-SC-0512. During the recycle period between CDDT 
and launch count, fan rundown tests were performed and indi cat ed ’satisfac- 
tory operation of all fans. 

TCP 0007 - Countdown - April 8 to April 11, 1970 .- At the beginning 
of the countdown, the cryogenic and fuel cell systems were in the following 
configuration: The fuel cell reactant systems were pressurized to 28 psia 

with reactant gas (left from the activation in CDDT to maintain system 
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Figure 3-5.- Liquid oxygen servicing (Launch complex 39). 
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integrity). The oxygen and hydrogen tanks -were pressurized to 80 psia 
with oxygen and hydrogen gas from the CDDT operations . The Ground Sup- 
port Equipment (GSE) lines were connected to the spacecraft and had been 
evacuated, pulse purged, and then pressurized with reactant gas to 80 psia. 
The pressure operated disconnects had been leaked checked at 80 psia with 
reactant gas. All samples taken from the GSE were within specification. 

The first activities began at 1500 hours on April 8, 1970. A dew 
point sample of both the LO2 and LH2 tanks were taken. Both systems met 
the 25 ppm requirement (oxygen tanks 1 and 2 read less than 2 ppm). Fol- 
lowing fuel cell activation and calibration, hydrogen and oxygen were 
serviced as in CDDT. Servicing was normal with the exception that LH2 
tank number 1 did not meet the 99 percent minimum loading requirement. 

98.7 percent was considered acceptable and DR SC 109-SC 0537 was closed 
by waiver number SC 109-PO-020. After servicing, a decay in the LO2 
tank number 2 pressure indicated that a leak existed (DR SC 109-SC 0538). 
All POD's were removed from both I0 2 tanks, and flight caps installed to 
stop the leak prior to pressurization. (Normal procedure is to disconnect 
and install the flight caps after pressurization. ) GSE heater power was 
supplied, the L0 2 tanks pressurized to approximately . 9^0 psia, and the 
hydrogen tanks to approximately 235 psia. Fan motor checks were then 
made. Performance was nominal. POD's were then removed from the LH 2 
tanks and final leak checks performed on the spacecraft POD's. No leaks 
were found. 

After L0 2 servicing fuel cell number 1 wets placed on Bus A (approx- 
imately 20 ampere load) to minimize the usage of LH 2 . (A constant flow 
from the tanks equal to the boil off rate results in minimum usage.) 

At approximately T-4 hours all fuel cells were placed on the busses, fuel 
cells 1 and 2 on Bus A and fuel cell 3 on Bus B. The cells then supplied 
the complete spacecraft load from this period through launch . 


3.3 DETANKING INCIDENT AND EVALUATION 


The. following chronology has been constructed from the best recol- 
lections of the personnel involved. No discussions could be found be- 
tween KSC personnel and other agencies specifically on the advisability 
of utilizing tank heaters for boil off. NR Downey personnel remember an 
awareness of this intent and of being so advised after the fact. Beech 
personnel have no such recollection. 
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3. 3-1 Description of the Problem 

On Monday, March 23, 1970, at 10 PM EST, following completion of 
the normal liquid oxygen loading sequence, console operators at KSC were 
unable to off-load LOX Tank ho. 2. The sequence at the time called for 
a reduction in quantity (both tanks) to approximately 60 % to provide 
astronaut breathing oxygen for the countdown demonstration test in pro- 
gress . Fluid was to be removed in the normal manner through the tank 
fill line by adjusting the facility gaseous oxygen expulsion pressure to 
80 psia, opening the fill line pressure operated disconnect valve (POD) , 
and permitting the tank to be pressurized through its vent line. 

In the ten-minute period required hy tank no. 1, the quantity in 
tank no. 2 had decreased to only 9&1 (fig. 3-7)- Several cycles were 
attempted to insure no stuck valves . 

Interim Discrepancy Report (IDR) No. 23 was then initiated. (An 
IDR is the normal method of officially documenting all anomalies found 
during testing when the problem cannot be specifically isolated without 
further trouble-shooting) . At that time, moisture was believed to be 
present in the ground support equipment (GSE) since ten previous space- 
craft had been successfully detanked. 

The NR and NASA console operators then requested permission from 
their respective supervisors to leave the oxygen in the tank, proceed 
with CDDT, and troubleshoot when the schedule permitted. 


3.3.2 Ground Support Equipment (GSE) Investigation 

Tuesday morning, March 2b, both NR and NASA local management were 
apprised of the situation. An engineering group was formed to plan and 
execute the troubleshooting. The tanking/ detanking procedure was to he 
reviewed, together with the moisture analysis sample reports; the inlet 
line filter was to be removed; X-rayed for a possible collapsed internal 
element, and then returned to the vendor (Wintee) for analysis; the POD 
was to be checked for proper actuation, both at ambient and at cryogenic 
temperatures; and the pad crew was to verify both the controlling solenoid 
valve and its pneumatic pressure supply. NR-KSC engineering pers<?nnel 
reported that the record hooks, reviewed prior to the flight readiness 
review (FRR) showed nothing significant. The Downey ’’shelf drop" DR in 
the cryo shelf record book, transferred from spacecraft 106 , was not 
remembered. 

By Tuesday evening, both the POD and the filter had been discon- 
nected from the lines on the Mobile Service Structure (MSS) and returned 
to the Malfunction Detection Laboratory. The filter had been x-rayed and 
sent to Wintee, and pad personnel had assured proper operation of the 
pneumatic supply, the POD actuation valve, and all associated flex hoses. 
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Wednesday, March 25, the filter x-rays were reviewed and showed no 
anomalies, the POD operation was checked and' proved normal, and no 
moisture or procedural problems could be. found. NASA-KSC subsystem 
personnel informed their counterparts at MSC of the (BE troubleshooting 
plan and its status. 

NASA/KSC Chief Project Engineer for CSM discussed the situation in 
the daily phone call with his MSC counterpart, the Chief CSM Project 
Engineer. • 

Wednesday evening, Wintec (the filter vendor) verbally reported that 
the filter had no restrictions and that there was no unusual contamination. 


3-3-3 Spacecraft Investigation 

Thursday morning, March 26, KSC, NR and NASA management -were advised 
that the problem was most probably either an open or a blockage in the 
spacecraft portion of the fill path. 

By Thursday afternoon, both the subsystem personnel at MSC and 
NR/Downey, and the project engineering management personnel at all loca- 
tions had been informed. Discussions were underway to attempt the de- 
tanking with higher expulsion pressures, with the possibility of having 
to "boil off" under consideration. 

The Beech Aircraft Company Program Manager was at Downey, and the 
Beech factory engineers were alerted. . Beech engineers began reviewing 
drawings and concluded that a gap in the dog- leg portion of the tank fill 
path (due to machining- tolerances and cryogenic temperatures) was indeed 
plausible. Such a gap could he approximately equivalent in area to the 
cross-section of the actual fill path (0.091 sq. in. vs. O.O98 sq. in.). 
Reference Beech Report MR15230 , dated April 2, 1970 . 

Friday morning, March 27, KSC, NR and NASA management personnel were 
briefed on the initial troubleshooting plan: 

The flight half of the POD was to be inspected as well as possible 
to insure no blockage due to seals or parts out of position; the fill and 
vent lines were to be conected; Tank No. 2 was to be vented through its 
fill line with the pressure existing in the tank at that time (both tanks 
had been partially pressurized for use during CDDT) ; nominal expulsion 
pressure 80 psia was then to be applied to both tanks, to empty Tank No. 1 
and to assure that the problem was repeatable in Tank No. 2; higher ex- 
pulsion pressures were to be attempted if required; and if that failed, 
the fluid was to be boiled off with tank heaters . 



42 


NR Downey Specification MA0201-3092 limited the application of 
heater power only if the tank was not serviced with cryogenics or in- 
ternally cooled. It also stated that cooling was necessary to insure 
thermal switch closure, and that the later series tanks, without thermal 
switches, could he damaged by overheating. 

In Houston, the MSC Chief CSM Project Engineer held a meeting with 
the fuel cell and cryogenic subsystem personnel. A review of tank change 
out problems revealed that the cryo shelf would have to be removed and 
that the available Downey GSE would not fit within the MSS clam shells . 

This meant a possible return of the space vehicle to the VAB and maybe 
demate of the spacecraft from the launch vehicle. Tank x-ray was dis- 
cussed. MSC personnel remembered a similar detanking problem on one tank 
of house spacecraft 2TV1. It was decided to dissamble both tanks, then 
at Downey. 

At Beech Aircraft Co. , personnel had contacted the Quantity Probe 
Vendor (S imm onds ) and had obtained circuit information. A test was set 
up and run indicating that a probe short (plat e -to -p late , possibly by a 
loose inconel tube in the dog-leg portion) would produce only 7.4 x 10 3 
joules (Beech Report MR1523G). The tolerance analysis had been completed 
showing a possible .008" interference fit where the inconel tube entered 
each Teflon piece. The stress levels appeared acceptable. A quick test 
for brittleness was conducted at cryogenic temperatures . No damage occured. 

By 2 PM EST, the MSS had been returned to position on the launch pad 
(it was moved to the park site for CDDT T-0) and the cryogenic tank fill 
and vent lines connected. With Downey personnel monitoring from their 
mission operations room via the operational intercom system (OIS), the 
tank at 178 psia was vented through its fill line (fig. 3-8). Onboard 
quantity decreased from 84$ to 65 %. Pad personnel viewing the vent out- 
let on the MSS did not see white vapor unusually present when liquid oxy- 
gen is expelled. Further troubleshooting was delayed until the normal 
post CDDT GSE purge could be completed. 

Friday afternoon, KSC NR engineering personnel discussed-the data 
with NR Downey subsystem personnel and agreed on the higher expulsion 
pressures to be used in the next attempt. It was understood by NR-KSC 
that if pressure would not do the job, then the fluid would have to he 
boiled off. KSC-NASA engineering personnel contacted Beech Aircraft 
to relay the same data, and found that it was already there. Beech 
personnel described the internal construction of the tank (no detail 
drawings were available at KSC), the fill path gap theory, the filter 
capability in the tank supply line (in case there were loose pieces of 
Teflon), and requested KSC elevation data since they were concerned about 
a possible fluid head problem. The possibility of detanking by heater 
boil off was discussed briefly. 
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Figure 3-8. - Oxygen tank 2 detanking characteristics (pressure decay through the 

fill line- CD DT). 



The KSC-NR Chief Engineer and the NASA CSM Chief Project Engineer 
received a joint call from the MSC MCE Apollo Spacecraft Program Office 
to discuss the tank change-out problem. The call was then transferred to 
the office of the KSC Launch Director and enlarged to include the MSC 
Chief CSM Project Engineer, MSC and ESC NASA subsystem personnel, the 
NR/Downey Chief CSM Engineer and his subsystem personnel, and KSC-NASA 
spacecraft operations management. 

The general situation was discussed including the fill path gap 
theory, considerations of loose Teflon pieces and the inconel tube, the 
implications of these during launch count and flight, and the problems 
involved with x-raying and changing a tank. To x-ray, the rather bulky 
camera would have to be positioned well inside the bay, with no stands 
available either for equipment or personnel. To change a tank the cryo 
shelf would have to be removed. With no GSE, a small technician would 
have to climb in behind the hydrogen tanks to manhandle the back side 
until the 2h'f lb shelf could be properly held by personnel in front. 

The consequences of damage resulting from. either of these operations 
were of much concern. 

It was decided to postpone the tank "keep vs change" decision until 
later; that KSC would continue in its efforts to detank, and that NR- 
Downey would expedite the 2TV-1 tank disassembly over the Easter weekend. 


3.3.^ Tank No. 2 Detanking Procedure 

Ry 20:00 EST Friday evening, March 27-, 1970 » the GSE had been 
purged and was ready for further detanking attempts. It must be re- 
membered in the following discussion that the vent POD's of both tanks 
(OV #1 and 0V §2) tie to a common manifold with the capability to switch 
either into an adjustable pressure source or to the facility vent system; 
also that these POD valves are designed to seal pressure into the tank 
when closed, but will not hold pressure in the reverse direction. 

First, the GSE regulator controlling the vent line pressure was 
adjusted to its detanking set point. Pressure in tank no. 2, at approx- 
imately 36 psia, increased to 90 psia (fig. 3-9) due to back flow through 
0V2, while tank no. 1 with 250 psia left over from CDUC remained stable. 
At 20:25:^2 QF1 and 0F2 were opened. Tank no. 1 began to empty while 
tank no. 2 received the inflow from the higher manifold pressure. Twenty 
seconds later 0V1 and 0V2 were opened and both tanks equalized while tank 
no. 1 continued dumping. Seven minutes later, tank no. 1 was empty and 
an 8 minute GOX purge was begun. Blowing through both tanks, the vent 
line pressure stabilized at 75 psia. 
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Figure 3 - 9 . - Characteristics of oxygen tank 2 detanking using GSE pressure (CDDT). 
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At 20:4l valves to tank no. 1 were closed and a purge was begun 
on tank no. 2 . Fourteen minutes later the inlet pressure was adjusted 
upward to 97 psia. Eleven minutes later the inlet pressure was again 
adjusted upward to approximately 120 psia. By 21:17 after a total of 
3 6 minutes of purging, the quantity had only decreased from 65 $ to 6l%, 
and it was obvious that something else would haye to be tried. 

The GSE regulator was then adjusted downward, 0F2 closed, and the 
tank allowed to pressurize to 190 psia. 0F1 was then opened and the 
tank allowed to vent. The quantity showed no appreciable change. At 
21:26 the process was repeated with a pressure of 222 psia. The results 
were similar. 

At 21:35 the NR and NASA engineers on station elected to try the 
previously planned heater "boil off" operation. The vent manifold 
supply pressure was secured and the tank allowed to bleed down through 
its fill POD. This produced an apparent quantity decrease of 5%. At 
21:47 0F2 was closed and, the heaters were turned on. At 21:52 the back 
flow pressure that had accumulated in tank no. 1 was vented. 

At Houston, the Acting MSC Subsystem Manager had been established 
as a communications point for the MSC MGR Apollo Spacecraft Program Office, 
and called the ESC NASA engineer on duty in the control room. He was in- 
formed of the lack of progress and given the heater on times and currents. 
He then suggested that the fans be utilized to add more heat and mixing. 
This was accomplished by 23:20 (fig. 3-10), and by 00:55 the tank temper- ' 
ature had reached upper limits ( 84 ° F) . 

Third shift personnel, both NR and NASA, (not scheduled) were called, 
and relieved second shift personnel approximately one hour later. From 
the plotted data (quantity vs time) the boil-off rate appeared to be de- 
creasing. The NR systems engineer on station remembered a procedure used 
by Linde Air Products whereby warm gas under pressure would add heat 
faster. The NR test project engineer, the NR system specialist, and the 
NASA test conductor on station were consulted. The lead NR systems en- 
gineer was called at home . A one hour wait was advised to confirm the 
rate decrease and to research specifications. The MSC Test Specifications 
and Criteria Document (TSCD) was reviewed for any system limitations. 

None were found for this abnormal operation and it was concluded only that 
the pressure must remain below 25 $ design burst ( 1515 .psia) -so as not to 
count tank cycles or present a safety problem. 

At 03:38, with the boil off rate decrease apparently confirmed, the 
vent manifold supply pressure was turned on, the tank pressurized to 
approximately 240 psia, and 0V2 closed at 03:55- Pressure in the tank 
continued to build up indicating heat transfer, and seven minutes later 
at 290 psi, 0F2 was opened and the tank allowed to vent to 60 psia. The 
seventeen minute venting process produced an apparent quantity decrease of 
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Figure 3-10.- Characteristics of oxygen tank 2 detanking using fans and heaters (CDDT). 



7$ (from 35 % to 28 %). The results were considered excellent and the tank 
was emptied with four additional cycles . Tank behavior and valve cycles 
can be found in figure 3-11 • V 

Fans were turned off when the quantity reached . 5 %. Heaters were 
turned off per the normal detanking sequence (29-019 of TCP-K-0007) that 
applies a minimal voltage to assure heater thermal switch closure when 
the temperature is below -75° F* 


3.3.5 Management Actions and Launch Rationale 

Saturday morning March 28, 1970, KSC-NR systems personnel related the 
details of the detanking procedure (pressures, quantities, heater on-off 
times, etc.) to NR-Downey systems personnel. 

In Houston, the MSC Chief CSM Project Engineer constructed a check- 
list of items required for the Monday decision point and called the NR- 
Downey Chief Engineer for CSM: 

1. The 2TV-1 tanks were to be opened and the parts flown to 
Houston. 

2. All probe materials were to be reviewed and a hazard analysis 
conducted, 

3. An investigation was to be made into the feasibility of 
x-raying the tank. 

4. Information was needed on how detanking was performed at Beech 
Aircraft . 

5- Downey and Beech engineering personnel were to understand how the 
detanking was accomplished. 

6. The problem of a restriction vs. a leak in the fill path was to 
be thoroughly analyzed. 

In Downey three tanks were x-rayed (XTA0019 and the two from space- 
craft 2TV-1). The x-rays showed only faint shadows where the inconel 
fill tube in the dog- leg was located. Neither Teflon piece was visible. 

The Beech Program Manager remained in Downey while the tanks were 
disassembled. Saturday afternoon KSC-NR and NASA project engineering 
personnel discussed a tank comparison blowdown test with MSC and Downey. 






Figure 3-11 Characteristics of oxygen tank 2 detanking using fans, heaters, and GSE pressure (CDDT1 
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Monday morning, March 30, MSC subsystem personnel received- the de- 
tanking details from KSC-UASA subsystem personnel. Beech engineers were 
also called and received the same details. In addition, the details of 
tank construction were further discussed together with the Beech detank- 
ing process. KSC engineers were told that Beech detanked at supercritical 
pressures and that no problems had been experienced. 

The UASA-KSC Director of Spacecraft Operations held a meeting with 
all involved -UR and NASA personnel in preparation for the afternoon 
decision conference. The problems involved with changing and X-raying 
the tank were discussed, together with a blowdown test, to eliminate 
line restriction considerations, and the possibility of a second tanking 
to demonstrate no launch count problems. 

At 13:30 both tanks were pressurized to approximately 250 psia and 
vented through the fill lines (tank no. 1 and then tank no. 2). With 
the exception of an apparent leak from tank no. 2 vent POD that caused a 
‘slight increase in tank no. 1 final pressure (this POD was capped for 
launch to stop the leak) both curves were essentially identical (fig. 3-12). 
Tank no. 1 dropped 135 psi, in bo seconds while tank no. 2 dropped lbO psi , 
indicating no permanent line restrictions . 

At lb: 00 a conference call was arranged betweeen the KSC Launch 
Director and the MSC MGR Apollo Spacecraft Program Office with NR-Downey 
participating. All aspects were discussed: The blowdown test showed no 

restrictions; the 2TV-1 probe parts appeared loose enough to match the 
fill line gap theory; the Beech tolerance/temperature analysis indicated 
a possible leak area approximately equivalent to the fill path cross- 
section; the quantity probe was then working and had ‘shown no previous 
anomalies which indicated no functional parts out of position; a loose 
inconel tube could not get out of the probe area (same exact diameter 
as the largest hole); shorting of the probe ( plat e-to-pl ate) would not 
present a risk since the energy was only 7-b x 10 3 joules); impact of 
Teflon parts by the fan impeller (shown improbable by Beech calculations 
and tests) was considered safe by similarity to a previous UR test/study 
conducted on a piece of neoprene assumed to be in an earlier spacecraft 
tank; the Beech detanking procedure as understood by both Downey and 
KSC personnel would not have indicated an open fill path; the ‘KSC de- 
tanking procedure was apparently understood by everyone; and .x-ray 
appeared impractical based on the UR experience and the inherent damage 
from personnel climbing inside the service module bay. 

The checklist developed Saturday was now complete. The shelf drop 
incident had not been remembered by engineering, and was unknown to 
management. The continuous heater on time (over 8 hours) had not been 
highlighted and the effects were not understood or considered at the time. 
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KSC was requested to demonstrate retanking capability to preclude 
a launch count risk, and to detank by the previously demonstrated suc- 
cessful method. If no anomalies occurred, then, based on the above 
rationale, the tank "would be accepted for launch. 

■The NASA Mission Director and the Apollo program Manager were con- 
tacted and concurred in the decision. 

At 15:50, 0 2 tank no. 2 was successfully filled to 17# with liquid 
oxygen. Detanking was accomplished over the next two hours (fig, 3-13) 
with five pressure cycles . Heaters remained on for approximately 32, min- 
utes . 


All subsequent operations were nominal. The tank showed no further 
peculiarities prior to launch. 


3,3.6 Post Launch Detanking Analysis 

With a heightened awareness of the KSC detanking procedural details, 
an investigation was conducted to positively settle the question of pre- 
launch tank damage. 

It was demonstrated that the quantity probe dog leg assembly could 
become dislodged with the correct tolerance conditions and thus prevent 
detanking; that the heater thermal switches would not open under ground 
power supply currents due to contact welding; and that the temperature 
developed due to continuous heater operation would produce severe fan 
wire insulation damage . 

It was further demonstrated that physical damage due to fluid 
effects was unlikely, 

MSC Heater Boil-off Testing .- A spacecraft fan/heater probe (no 
thermal switches) instrumented with 13 thermocouples, was placed in a 
full tank of liquid nitrogen (same approximate size as a flight tank). 
Liquid was then boiled off at ambient pressure by running the fans and 
heaters continuously. After six hours (fig. 3-14) with the fluid level 
still two inches above the lower fan , the surface temperature of the 
uppermost heater coil had stabilized at 1000° F. Directly behind this 
element , in the inside of the probe , the lower fan wiring is routed 
through a thin stainless .steel conduit touching the .020 inch heater 
probe wall. All fans and heaters continued to operate for the remaining 
four hours of the test, with the fan motor case temperature reaching 
some 350° F. 
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Post-test disassembly showed that the fan wiring had adhered to the 
conduit wall (some insulation was torn in its removal); that the insula- 
tion had changed composition and was very hard; and that complete circum- 
ferential cracking had occurred producing segments of insulation not over 
3A-inch long for a six inch lengbh extending downward from the hottest 
portion. 

An attempt was made to repeat the test with thermal switches and a 
55 percent full tank (more closely simulating KSC conditions). Upon 
first power application, one thermal switch was found open. Both switches 
were then by-passed, with power application established by the cycling of 
the good switch. A pattern was established that showed initial opening 
after eleven minutes and then a three minute on and a three minute off 
cycle. During this period both switches operated normally, and after 
four off cycles, it was decided to again place both switches in series 
with their heater elements. Neither switch opened again. The test was 
terminated after approximately 1-1/2 hours . 

Post-test disassembly showed the contacts had become fused together. 
Subsequent review of specifications revealed that the switch was rated 
only for 30 V dc and not 65 V dc , and that the . 015 inch contact separa- 
tion would not permit the opening arc to extinguish. 

MSG Conduit Test (2P2 iUM) .~ As a result of Beech computer studies 
of the tank exit conduit, showing possible 440° F wire temperatures, 
spacecraft hardware was obtained and placed in a chamber under the cor- 
rect vacuum and electrical load conditions . Ambient- temperature oxygen 
was supplied and the KSC sequence duplicated with regard to all influ- 
encing elements . Maximum conduit temperature reached 326° F after ap- 
proximately five hours. The pressure cycling produce no appreciable 
effect. Since this is a worse case boundary, it is concluded that tank 
S/N 0008 conduit wiring was not thermally overstressed prior to launch. 

Fluid Dynamics Effects .- The damage potential of high velocity fluid 
was investigated with regard to the internal parts of the density probe 
(the center supporting rivet was established as the weakest element) and 
also with regard to its effect on the upper dog-leg assembly. The rivet 
failure point was established by static load test. 

Liquid During Detanking .- The following analyses do not preclude the 
possibility of small slugs of liquid but, for the period investigated, 
they do show that the average flow could not have been liquid and that 
it was most probably gas . 

From figure 3-15 the maximum detanking rate occurred during the 
third venting. From the quantity gage readout over the six minute pe- 
riod, the flow rate was 0.07 lb /sec. Since the fluid quantity flowing 
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cannot change for the length of the path, and if we assume the hole in 
the top of the stand pipe then: 


W . = W . + W. _ 

pod prohe hole 


or 


AVp 


pod 


- AVp 


hole 


AVp , 
prohe 


The highest possible velocity, and consequently the worst damage 
potential, will occur if there is no hole. Eliminating the hole, Vp of 
the media calculates to 30.2 lh/ft^/sec, and using the density of liquid 
oxygen at 180 psia, the velocity of fluid in the standpipe must he 
0.5 ft/sec. This requires a driving pressure of only 0.0016 psi , and 
while the actual pressure cannot be established, it is known to be in' 
excess of 100 psi. Prom this it can be seen that the average density 
must be appreciably lower, and that the exit media could not have been 
liquid over the 6 minutes examined. 

Prom- figure 3-12, the blowdown test conducted at Kennedy Spacecraft 
Center, and using the tank volume with the measured pressures, the flow 
rate and media density at the time were 0.09 lbs/sec and 1.1 lbs/ft 3 . 
During the venting period discussed above, the flow rate was 0.07 lbs/sec. 
Since velocity is a function of pressure, and since pressure is controlled 
by friction line losses, then velocity becomes a function of viscosity 
and flow rates. The viscosity of oxygen does not change appreciably with 
temperature, and since the flow rates were approximately equal, it then 
follows that the velocities in both cases were approximately equal. 

Since the same tank vented thru the same lines in both cases, and 

since 


W = AVp 


it follows that the densities were approximately equal. Prom this it is 
concluded that the exit media was most probably gas. 

Liquid Damage to the Probe Rivet .- The following analysis review's 
the possibility of liquid damage to the probe rivet (worst case condi- 
tions ) . 



58 


Assuming liquid flow, the maximum pressure differential used, and 
assuming no significant velocity in the tank hulk fluid, then, for the 
configuration shown in figure 3 - 16 , the velocity will he 238 ft /sec. 
Computing a Reynolds number to establish a drag coefficient of 1.2, the 
force on the rivet can he calcinated to he 63. 0 pounds. 

The actual force seen hy the rivet was, of course, significantly 
less since there was an appreciable leak between the tank and the fill 
path at the upper end. The assumed 320 psi delta could not have existed. 
If a pressure of even 100 psi could have been sustained, then the force 
would drop to as little as 11 pounds , The actual failure point of the 
rivet was determined to be 105 pounds by static load (reference MSC test 
Mo. TPS-JL3-T-60 ) . 

Torque on Probe Dog -Leg .- This analysis shows that the normal fluid- 
produced torque on the probe dog-leg assembly would be sufficient to dis- 
lodge ‘it under the correct manufacturing tolerance conditions (fig. 3 -I 7 ). 

Assuming the maximum pressure differential that would occur during 
normal detanking (as the fluid interface rises to contact the first bend) 
(fig. 3 - 18 ) and with no velocity in the tank bulk fluid, then the stand 
pipe velocity is calculated to be 139 ft /sec. Equating the upward force 
to the sum of the momentum force and the pressure force, F u is calcu- 
lated to be 58.0 pounds, with the moment 32. k in-lbs. Utilizing meas- 
ured flow rates during a normal detanking, the moment can be reduced to 
5-3 in-lbs. This is still adequate to produce motion. Detanking tests 
at Beech Aircraft have shown that fluid will be expelled in the worst 
case short (fig. 3 - 17 ) only if the assembly is exactly aligned. 

Thermal Effects . - As a result of Beech computer studies of the tank 
exit conduit, showing possible F wire temperatures, and a drawing 

study that permitted the lower fan wiring to cross a heater element, tests 
were organized at Manned Spacecraft Center and at Beech to reproduce the 
Kennedy Spacecraft Center detanking procedure . 


3.^ SUPPORTING DATA 


The significant documentation, discrepancy reports (DR r s), Material 
Review Board Actions (MR's), Specifications, Test Preparation. Sheets 
(TPS's), Related Documents, and Waivers, necessary to support, the con- 
clusions drawn in section 3.2 and 3.3 axe tabulated in this section. 
These items may be obtained from the spacecraft records or applicable 
NASA offices. 
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Figure 3-16.- Density probe cross 


section showing fluid force on the rivet. 
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Manufacturing tolerance study. 
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Figure 3-18.- Standpipe assembly at the top of the tank. 
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3.4.1 DR's, MR's, Spec! fi cat i on , TPS's, Related Documents,' 

Waivers, and TCP's 


PR's .- 

SC 109-SC0154 
SC0118 
SC0121 

sco 325 

SC0366 

SCO 37 3 

SCO 374 

SCO 380 
SC0387 

SC0441 
SCO 512 

SCO 521 

SCO 537 

SC0538 


- Remove and replace all fuel cells 

- Oxygen tank Wo. 1 vac -ion, pump converter replaced 

- LH 2 tank No. 2 pressure transducer failed. Replaced 
per TPS SC042 

- Remove and replace LH 2 tank No. 1 

- Fuel cell No. 1 LH 2 regulated pressure transducer 
replaced. 

- Fuel cell No. 2 connector P62 has recessed pin. 
Condition repaired. 

- Fuel cell No. 2 connector J1 has anodizing off on 
first thread. Condition accepted. 

- Remove and replace fuel cell No . 1 

- Fuel ce ll No. 1 did not load share properly. Fuel 
cell removed and replaced per above DR. 

- Remove and replace fuel cell No. 2 and No. 3 

- Spacecraft could not be detanked with normal detank 
procedures. Condition accepted for flight. 

- Oxygen tank No. 1 quantity indication showed random 
fluctuation. Condition accepted for flight. 

- LH 2 tank No. 1 quanity accepted for flight (waiver 
No. 109PO-020). 

- LQ 2 tank No. 2 vent POD leaked. No leakage with 
flight cap installed and condition accepted per MR. 


DR GSE 132-1-1735 - GSE investigated to insure no restrictions during 

detanking problem. 


MR's. - 

MR Action - Replacement of fuel cells (reference DR SC109-SC0154 ) 

MR Action - Acceptance of L0 2 tank No. 2 vent POD leak (reference DR 
SC109-SC0538) 

Specifications . - 

MA0201-3092 - Cryogenic gas storage system checkout and servicing 
requirements - KSC. 

MA0201-3089 - Cryogenic storage system checkout requirements - Downey 

SN9-R007B - Test and checkout requirement document for KSC - 

CSM 108 and subsequent vehicles, dated December 31 s 19&9- 

SPT-0003A - Test and checkout specification and criteria document 

for KSC (CSM 108 and subsequent CSM's), dated 
February 1970 . 
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TPS 1 s . - 

TPS GS-088-1-473 *- MSOB facility activation 

S/C-109-SC042 - Replace LH 2 pressure transducer 
S/C-109-SC045 - Retest of LH 2 pressure transducer 
S/C-109-SC118 - Remove SM-2 for LH 2 tank No. 2 replacement 
S/C-109-SC119 - Remove and replace LH 2 tank No. 2 

Related Documents ." 

1. Apollo fuel cell and cryogenic gas storage system flight support 
handbook . 


2. Beech test procedure No. BP-14346-1, dated 6 / 6 / 66 , Rev. A, 

10-18-66. 


3. Beech memorandum report MR 15230, dated April 2, 1970. 

4. NR letter 68MC197 (contract No. NAS 9-150, 2TV-1 SESL LOX dump 
system analysis) to NASA MSC, dated July 29, 1968 . 

5. SD-68-609, engineering analysis report, thermal -vacuum test 1 , 
spacecraft 2TV-1, dated August 15, 1968 . 

6 . 2P2l4M - MSC test report on the test to - determine the temperature 
rise in the conduit during the modified detanking on S/C 109 at KSC. 

7. Wintec Corporation test report TR-220 on hardware analysis of 
P/N 15241-637 Wintec filter assembly S/N 001. 

8 . S/C 2TV-1 test project engineering report, thermal vacuum test, 
June 24, 1968 . 


9 . CSM 2TV test project engineering report, thermal vacuum test 
No. 2 Septemper 9, 1968 


Waivers . - 

109-P0-001 - Musty odor in the suit loop samples. 

109-PO-020 - LH 2 tank No. 1 quantity of 98.7 percent did not meet 
the 99 percent requirement of SP0003B. 


TCP r s 
TCP 3063 

TCP 3071 

TCP 0070 

TCP 0048 

TCP 0034 


- Service module move and' mate preps 

- CSM mate 

- Abbreviated combined systems test 

- Simulated altitude chamber test 

- Altitude chamber test 
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TCP 005 - Integrated test with launch vehicle simulator 

TCP 0028 - Plight readiness test 

TCP 0007 - Countdown demonstration and countdown 

TCP 5127 - EES water glycol servicing and compressibility 


TCP4734 - Pad 39 facility activation for the L0 2 system 
TCP 4773 - L0 2 and LH 2 dewar servicing 


3.4.2 Sampling and Loading Procedures, Pill Line Filter Analysis 

The purity of a test complex, used for gas leak and functional checks 
is normally certified for use by gas purity and particulate samples fol- 
lowing its original program, activation flushing. Between certification 
and use, a gas pressure pad is left in the system. Presence of this pad 
pressure maintains systems integrity. 

MSOB gas sampling .- The cryogenic and fuel cell fluid distribution 
system (FDS) in the MSOB (reference fluid schematic -for MSOB gas sup- 
plies, fig. 3-3) was validated for spacecraft 109 per TPS-088-1-473 
prior to any spacecraft testing. The leak checks and sampling per this 
TPS were performed with gaseous nitrogen. The gaseous helium purity used 
to pressurized the cryogenic system in TCP-3071 and TCO-0070 was insured 
by facility samples upstream of the FDS. Gaseous oxygen used to pressur- 
ize the oxygen cryogenic system in TCP-0048 and TCP-0034 was from certi- 
fied (QC controlled analysis by the supplying agency) K-bottles . ' The 
oxygen content was verified with a Beckman analyzer at the K-bottle out- 
let and a purity sample was taken from inside the command module at the 
suit loop outlets. All samples were within the required specifications 
with the exception of some musty odor from the suit loop sample. This 
odor is normal, detected on all spacecraft, with the required specifica- 
tion being changed to reflect this . 

MSOB filters .- The GSE valve box and the spacecraft interface' filter 
for the MSOB system are 5 micron nominal - 15 micron absolute. The fil- 
ters were removed from the system, cleaned, and bubble point verification 
performed at the beginning of TPS-088-1-473- The filters were then 
reinstalled in the system under a GN 2 purge . 

LC-39 gas sampling .- The cryogenic and fuel cell fluid distrubution 
systems at LC-39 (fig- 3-4) were verified by TCP-4734 prior to any space- 
craft testing. The gaseous helium purity, used to perform the Pressure 
Operated Disconnect leak checks , was insured by facility samples upstream 
of the EDS. The gaseous oxygen used at LC-39 to pressurize the cryogenic 
system came -from a certified gaseous oxygen tube-bank trailer. The 
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oxygen purity was verified "by samples taken at the filter sample valve 
at the oxygen purge disconnect. All samples were within the required 
specifications . 

LC-39 filters .- The filters on the oxygen vent and fill disconnects 
are 15 micron absolute Wintec filters. The filter at the oxygen purge 
disconnect is a 5 micron nominal - 15 micron absolute Capital Westward. 

Liquid oxygen loading .- The spacecraft liquid oxygen tanks are 
serviced from a portable dewar. The dewar is serviced from a certified 
portable t ank er through a 5 micron-15 micron absolute filter at a fluid 
complex (Cryo II Building) with the liquid oxygen overflow sampled for. 
purity at the outlet vent of the dewar during filling (reference dewar 
servicing schematic, fig. 3-6). The sample is a liquid sample. The 
dewar is then transferred to the pad by truck and connected directly 
into a fill manifold already connected to the spacecraft, (fig. 3-5)* 
Prior to servicing, the spacecraft tanks are evacuated to 5 torr and 
held for a period of two hours. After the vacuum break with gaseous 
oxygen, a moisture sample is taken from the tanks (valve G35MV-2) and 
verified to be less than 25 ppm. Pans are turned on immediately prior 
to servicing and turned off at approximately 100 percent load. Liquid 
oxygen is then pressure transferred (^5 psia Gox) into both tanks simul- 
taneously at a flow rate of approximately 25 pounds per minute and al- 
lowed to overflow into the vent line system for 10 minutes after the 
tanks have filled. Valving is then closed trapping the warming liquid/ 
gas media between the spacecraft vent disconnects and the facility vent 
valve. This is allowed to expand through valve G35MV-2 into a sample 
container and is analyzed for helium, nitrogen, and total hydrocarbons. 
All samples taken during liquid oxygen servicing of spacecraft 109 were 
within specification. 

At the time of the initial detanking incident in CDDT (reference 
section 3-3) the filter for oxygen tank 2 fill line was removed to in- 
vestigate for possible restrictions. 

When the Wintec Filter was examined by the Wintec Corporation, 
three glass beads were found in the effluent that was flushed through 
the filter in the reverse direction (from spacecraft to GSE). There 
was also a quantity of lubricant found on the GSE side of the filter 
assembly, identified as "Krytox 2^0 AC". This lubricant (L0 2 compatible) 
is normally used at KSC during the connection of the close-tolerance 
bayonet fitting to its filter assembly. Samples of the effluent when 
the filter was flushed in the flow direction, from GSE to spacecraft, 
contained 13 particles between 100 and 250 micron size and 9 particles 
greater than 250 microns. These were identified only by metalic color 
as copper, brass, bronze, copper-gold, and some unidentified fibers and 
plastic. Most of these particles presumably came out of spacecrafts 
during S/C 108 and S/C 109 detanking, and from the disconnection of the 
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filter and pressure operated disconnect. It, of course, cannot be de- 
termined exactly when these contaminants entered the filter or from 
which spacecraft; however, the number of particles is not considered 
significant based on the quantity of fluid flowed through the filter 
during spacecraft detanking and the 8.5 gallons of fluid used to obtain 
the samples. The spacecraft cleanliness level allows 10 particles be- 
tween 100 and 175 microns per 500 ml of flush fluid (this would equate 
to 6J+0 particles for the quantity of flush fluid used). 

The three glass beads were 15, 45, and 50 microns in size. It can 
be concluded, based on the bead size, that the beads did not come from 
the spacecraft tanks since the Wintec bubble point of the filter element 
correlated to an equivalent pore diameter of 9*4 microns. Investigation 
into the source of the beads has not been positive. Based on the small 
size of any beads that could have passed through the filter into the 
tank (approximately 0.0004 inches) it is concluded that the beads could 
not have been detrimental to the system's performance. 



3. 4. 2.1 Summary of KSC Sample Results 


Author! zing Doc . 

Sample Origin 

Specification 

Fluid/Sample Type 

Within 

Specification 

TPS-088-1-473 (MSOB ) 

Outlet of filters on GSE at 
oxygen fill (l and 2 ) and 
oxygen purge 

MSC-SPF-0021 

ffl} 2 /Part i culat e 

GH 2 /Purity (Hydocarbons only) 

Yes 

Yes 

TCP-3071 

Facility GHe Battery Hi and 
HZ 

MSFC-364A 
(Meets -364 b) 

GHe/Purity 

Yes 

TCP-0070 

Facility GHe Battery #1 and 
HZ 

MSFC-364A 

GHe /Purity 

Yes 

TCP-0048 

Inside CM - suit loop. 

SPT-003A 

(TSCD) 

G0 2 /Purity (For this test 
certified K-hottles of G0 2 
are utilized) 

Yes*’ 

TCP-003 1 * 

Inside CM - suit loop 

SPT-0003A 

(TSCD) 

G0 2 /Purity (For this test 
certified K-bottles of G0 2 
are utilized) 

Yes* 

TCP-4734 

GSE - Filter sample valve 
at oxygen purge filter 

MSC-SPF-0021 

GN 2 /ParticUlate (Using facility 
supply) 

Yes 


0 2 tank vent line sample 
valve G35MV2 

M3C-SPF-0021 

GHe/Parti culate 

Yes 


0 2 tank fill flex hose 
outlet (G35FHl44l) 

MSC-SPF-0021 

G0 2 /Particulate (Using G0 2 from 
certified K-hottles ) 

Yes 

TCP-0005 

GSE - Filter sample valve 
at oxygen purge filter 

MSC-SPF-0021 

G0 2 /Purity 

Yes 


Facility GHe Storage 

MSFC-364A 
(Meets 364 b) 

GHe/Purity 

Yes 

TCP- 00 28 

GSE — Filter sample valve 
at oxygen purge filter 

MSC-SPF-0021 

G0 2 /Purity 

Yes 


Facility GHe storage 

MSFC-364A 
(Meets 364 b) 

GHe /Purity 

Yes 


Waiver granted for musty* odor. 


c\ 
— i 



3. ^.2.1 Summaiy of KSC Sample Results - Concluded 


o\ 

co 


Within 


Authorizing; Doc. 

Sample Origin 

Specification 

Fluid/Sample Type 

Specification 

TCP-0007 

- CDDT 

GSE - Filter sample valve 
at oxygen purge filter 

MSC-SPF-0021 

GO^/Purity 

7es 



Moisture-Gas flow from the 
tank and sample at sample 
valve G35MV2 

TSCD - 25 ppm 

GO 2 /Moisture 

yes 



GSE - Vent line sample 
valve G35MV2 

TCH-256 (TSCD) 

GO 2 /Purity 

yes 

TCP-0007 

-Recycle 
from CDDI 

GSE - Filter sample at 
oxygen purge filter 

MSC-SPF-0021 

GO 2 /Purity 

Yes 

TCP-0007 

- CD 

Moisture - Gas flow from 
the tank and sample at 
sample valve G35MV2 

TSCD - 25 ppm 

GO 2 /Moisture 

Yes 


■ 

GSE - Vent line sample 
valve G35MV2 

TCN - 256. (TSCD) 

GO 2 /Purity 

Yes 

TCP- 4773 

- CDDT 

GSE - LO 2 Dewar vent line 
sample valve during Dewar 
servicing (S/N-2) 

MSF-SPF-0021 

L0 2 /Purity 

Yes 

TCP- 47 73 

- DR 512 

GSE - LO 2 Dewar vent line 
sample valve during Dewar 
servicing (S/H-l) 

MSC-SPF-0021 

W 2 /Purity 

Yes 

TCP-4773 

- CD 

GSE - LO 2 Dewar vent line 
sample valve during Dewar 
servicing (S/N-2) 

MSC-SPF-0021 

LO 2 /Purity 

Yes 
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3. 4. 2. 2 Sample Analysis Report Numbers 


Reference Document 


Lab Sample No’s 


TPS-08801-473 

G22497 

G 22 i *98 

G22499 

G22500 

G22501 

G22502 

TCP -3071 

G22940 

G22337 

TCP -0070 

G23997 

G24645 

G25152 

G25654 

TCP-0048 

G29 2l4 
G29387 

TCP-003 1 * 

G29880 

G29881 

TCP-4734 

G38502 

G38503 

G38500 

G38501 

G38509 

G38507 

TCP-0005 . 

G00261 

G00262 

G39747 

G00301 

TCP-0028 

G01908 
G01909 
GO 0891 
G01446 
GO 20 09 

TCP-0007 -CDDT 

GO 6l60 
G06l6l 
G06113 
G06114 

TCP-0007-Recycle 

G 06103 

go6io4 

TCP-0 007-CD 

G07395 
GO 7 386 



TO 


TCP-ii773-CDDT 

G06097 

G06098 

TCP-4TT3-DR512 

GO 679 5 
G06796 

TCP-UT73-CD 

G07377 
GO 737 8 


NASA — MSC — Coml., Houston, Texas 



